{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "U9i2Dsh-ziXr"
      },
      "source": [
        "# Eager Execution Tutorial: Importing Data\n",
        "\n",
        "This notebook demonstrates the use of the [`tf.data.Dataset` API](https://www.tensorflow.org/programmers_guide/datasets) to build pipelines to feed data to your program. It covers:\n",
        "\n",
        "* Creating a `Dataset`.\n",
        "* Iteration over a `Dataset` with eager execution enabled.\n",
        "\n",
        "We recommend using the `Dataset`s API for building performant, complex input pipelines from simple, re-usable pieces that will feed your model's training or evaluation loops.\n",
        "\n",
        "If you're familiar with TensorFlow graphs, the API for constructing the `Dataset` object remains exactly the same when eager execution is enabled, but the process of iterating over elements of the dataset is slightly simpler.\n",
        "You can use Python iteration over the `tf.data.Dataset` object and do not need to explicitly create an `tf.data.Iterator` object.\n",
        "As a result, the discussion on iterators in the [Programmer's Guide](https://www.tensorflow.org/programmers_guide/datasets) is not relevant when eager execution is enabled."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "z1JcS5iBXMRO"
      },
      "source": [
        "# Setup: Enable eager execution\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "RlIWhyeLoYnG"
      },
      "outputs": [],
      "source": [
        "# Import TensorFlow.\n",
        "import tensorflow as tf\n",
        "\n",
        "# Enable eager execution\n",
        "tf.enable_eager_execution()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "H9UySOPLXdaw"
      },
      "source": [
        "# Step 1: Create a source `Dataset`\n",
        "\n",
        "Create a _source_ dataset using one of the factory functions like [`Dataset.from_tensors`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#from_tensors), [`Dataset.from_tensor_slices`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#from_tensor_slices) or using objects that read from files like [`TextLineDataset`](https://www.tensorflow.org/api_docs/python/tf/data/TextLineDataset) or [`TFRecordDataset`](https://www.tensorflow.org/api_docs/python/tf/data/TFRecordDataset). See the [Programmer's Guide](https://www.google.com/url?sa=D\u0026q=https%3A%2F%2Fwww.tensorflow.org%2Fprogrammers_guide%2Fdatasets%23reading_input_data) for more information."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "WPTUfGq6kJ5w"
      },
      "outputs": [],
      "source": [
        "ds_tensors = tf.data.Dataset.from_tensor_slices([1, 2, 3, 4, 5, 6])\n",
        "\n",
        "# Create a CSV file\n",
        "import tempfile\n",
        "_, filename = tempfile.mkstemp()\n",
        "with open(filename, 'w') as f:\n",
        "  f.write(\"\"\"Line 1\n",
        "Line 2\n",
        "Line 3\n",
        "  \"\"\")\n",
        "ds_file = tf.data.TextLineDataset(filename)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "twBfWd5xyu_d"
      },
      "source": [
        "# Step 2: Apply transformations\n",
        "\n",
        "Use the transformations functions like [`map`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#map), [`batch`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#batch), [`shuffle`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#shuffle) etc. to apply transformations to the records of the dataset. See the [API documentation for `tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) for details."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        },
        "colab_type": "code",
        "id": "ngUe237Wt48W"
      },
      "outputs": [],
      "source": [
        "ds_tensors = ds_tensors.map(tf.square).shuffle(2).batch(2)\n",
        "ds_file = ds_file.batch(2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "IDY4WsYRhP81"
      },
      "source": [
        "# Step 3: Iterate\n",
        "\n",
        "When eager execution is enabled `Dataset` objects support iteration.\n",
        "If you're familiar with the use of `Dataset`s in TensorFlow graphs, note that there is no need for calls to `Dataset.make_one_shot_iterator()` or `get_next()` calls."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "base_uri": "https://localhost:8080/",
          "height": 153
        },
        "colab_type": "code",
        "executionInfo": {
          "elapsed": 388,
          "status": "ok",
          "timestamp": 1525154629129,
          "user": {
            "displayName": "",
            "photoUrl": "",
            "userId": ""
          },
          "user_tz": 420
        },
        "id": "lCUWzso6mbqR",
        "outputId": "8e4b0298-d27d-4ac7-e26a-ef94af0594ec"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Elements of ds_tensors:\n",
            "tf.Tensor([1 9], shape=(2,), dtype=int32)\n",
            "tf.Tensor([16 25], shape=(2,), dtype=int32)\n",
            "tf.Tensor([ 4 36], shape=(2,), dtype=int32)\n",
            "\n",
            "Elements in ds_file:\n",
            "tf.Tensor(['Line 1' 'Line 2'], shape=(2,), dtype=string)\n",
            "tf.Tensor(['Line 3' '  '], shape=(2,), dtype=string)\n"
          ]
        }
      ],
      "source": [
        "print('Elements of ds_tensors:')\n",
        "for x in ds_tensors:\n",
        "  print(x)\n",
        "\n",
        "print('\\nElements in ds_file:')\n",
        "for x in ds_file:\n",
        "  print(x)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "default_view": {},
      "name": "Eager Execution Tutorial: Importing Data",
      "provenance": [],
      "version": "0.3.2",
      "views": {}
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
